Papers with LLM architectures
Learning to Edit: Aligning LLMs with Knowledge Editing (2024.acl-long)
Copied to clipboard
Yuxin Jiang, Yufei Wang, Chuhan Wu, Wanjun Zhong, Xingshan Zeng, Jiahui Gao, Liangyou Li, Xin Jiang, Lifeng Shang, Ruiming Tang, Qun Liu, Wei Wang
| Challenge: | Existing knowledge editing techniques rely on memorizing updated knowledge, impeding LLMs from effectively combining the new knowledge with their inherent knowledge when answering questions. |
| Approach: | They propose a Learning to Edit framework that equips LLMs with the ability to apply updated knowledge to input questions through a two-phase process . |
| Outcome: | The proposed framework outperforms existing methods in knowledge editing tasks and compares it with four benchmarks and two LLM architectures. |
SEUF: Is Unlearning One Expert Enough for Mixture-of-Experts LLMs? (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in LLMs unlearning have shown remarkable success in removing unwanted data-model influences while preserving the model’s utility for legitimate knowledge. |
| Approach: | They propose a Selected-Expert Unlearning Framework (SEUF) that combines expert attribution and an anchor loss to ensure controlled unlearning. |
| Outcome: | Experiments show that the proposed framework improves forget quality and model utility by 35% on MoE LLMs across benchmarks and LLM architectures compared to standard unlearning algorithms . |
Investigating the Robustness of Modelling Decisions for Few-Shot Cross-Topic Stance Detection: A Preregistered Study (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing models for stance detection are not robust enough to be used in a viewpoint-diverse news recommender because the news constantly has new discussion topics. |
| Approach: | They propose to use two stance task definitions (Pro/Con versus Same Side Stance) and two LLM architectures (bi-encoding versus cross-encode) to test model performance. |
| Outcome: | The proposed models outperform the same side-stance definition and other models on stance across different topics. |
Slim-SC: Thought Pruning for Efficient Scaling with Self-Consistency (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that Test-Time Scaling (TTS) can improve reasoning performance without retraining the model. |
| Approach: | They propose a step-wise pruning strategy that identifies and removes redundant chains using inter-chain similarity at the thought level. |
| Outcome: | The proposed method reduces inference latency and KVC usage by up to 45% and 26% with R1-Distill while maintaining or improving accuracy. |
EULoInf: Efficient Hessian-Free Entropy Based Uncertainty-Aware Data Influence Approximation (2026.findings-acl)
Copied to clipboard
| Challenge: | Extensive studies show that the effectiveness of fine-tuning heavily relies on the quality of training data. |
| Approach: | They propose a framework that approximates influence via uncertainty and gradient based validation loss lookahead. |
| Outcome: | The proposed framework matches or outperforms prior methods across diverse tasks and LLM architectures while reducing computational time and memory usage by over 50%. |